Utility of Complex Alternatives in Multiple-Choice Items: The Case of All of the Above

This study investigated the utility of all of the above (AOTA) as a test option in multiple-choice items. It aimed at estimating item fit, item difficulty, item discrimination, and guess factor of such a choice. Five reading passages of the Key English Test (KET, 2010) were adapted. The test was reconstructed in 2 parallel forms: Test 1 did not include the abovementioned alternative, whereas Test 2, administered 2 weeks later, included such an alternative. The 2 tests, 32 items each, were administered to 142 high school third-graders. Results, analyzed through 3-parameter logistic model, indicated that the multiple-choice questions, including the alternative all of the above, were easier. Results also revealed that the option all of the above increased the guess factor. Because guess factor is a source of measurement error, it may threaten test validity and reliability.